{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "02_NumPy",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_9J7KmeNNPUY",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://practicalai.me\"><img src=\"https://raw.githubusercontent.com/practicalAI/images/master/images/rounded_logo.png\" width=\"100\" align=\"left\" hspace=\"20px\" vspace=\"20px\"></a>\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/numpy.png\" width=\"200\" vspace=\"30px\" align=\"right\">\n",
        "\n",
        "<div align=\"left\">\n",
        "<h1>NumPy</h1>\n",
        "\n",
        "In this lesson we will learn the basics of numerical analysis using the NumPy package.\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bl9ki0kQNhxm",
        "colab_type": "text"
      },
      "source": [
        "<table align=\"center\">\n",
        "  <td>\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/images/rounded_logo.png\" width=\"25\"><a target=\"_blank\" href=\"https://practicalai.me\"> View on practicalAI</a>\n",
        "  </td>\n",
        "  <td>\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/images/colab_logo.png\" width=\"25\"><a target=\"_blank\" href=\"https://colab.research.google.com/github/practicalAI/practicalAI/blob/master/notebooks/02_NumPy.ipynb\"> Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/images/github_logo.png\" width=\"22\"><a target=\"_blank\" href=\"https://github.com/practicalAI/practicalAI/blob/master/notebooks/basic_ml/02_NumPy.ipynb\"> View code on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K0FzU8_LTyKF",
        "colab_type": "text"
      },
      "source": [
        "# Set up"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0-dXQiLlTIgz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bhaOPJV7WA0m",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Set seed for reproducibility\n",
        "np.random.seed(seed=1234)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VoMq0eFRvugb",
        "colab_type": "text"
      },
      "source": [
        "# Basics"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wnn5IE_aBcpP",
        "colab_type": "text"
      },
      "source": [
        "Let's take a took at how to create tensors with NumPy.\n",
        "*    **Tensor**: collection of values \n",
        "\n",
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/tensors.png\" width=\"650\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "23tSlin9aWZ8",
        "colab_type": "code",
        "outputId": "0ebe0914-761b-40ac-87d7-9090e17a3a57",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Scalar\n",
        "x = np.array(6) # scalar\n",
        "print (\"x: \", x)\n",
        "# Number of dimensions\n",
        "print (\"x ndim: \", x.ndim)\n",
        "# Dimensions\n",
        "print (\"x shape:\", x.shape)\n",
        "# Size of elements\n",
        "print (\"x size: \", x.size)\n",
        "# Data type\n",
        "print (\"x dtype: \", x.dtype)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:  6\n",
            "x ndim:  0\n",
            "x shape: ()\n",
            "x size:  1\n",
            "x dtype:  int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ugIZprdIabFF",
        "colab_type": "code",
        "outputId": "d3c8cf34-2824-4515-d922-ca761102179f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Vector\n",
        "x = np.array([1.3 , 2.2 , 1.7])\n",
        "print (\"x: \", x)\n",
        "print (\"x ndim: \", x.ndim)\n",
        "print (\"x shape:\", x.shape)\n",
        "print (\"x size: \", x.size)\n",
        "print (\"x dtype: \", x.dtype) # notice the float datatype"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:  [1.3 2.2 1.7]\n",
            "x ndim:  1\n",
            "x shape: (3,)\n",
            "x size:  3\n",
            "x dtype:  float64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SQI-T_4MbE9J",
        "colab_type": "code",
        "outputId": "ba3b8e7d-a160-44c4-cae5-5b6eee3f3012",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "# Matrix\n",
        "x = np.array([[1,2], [3,4]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x ndim: \", x.ndim)\n",
        "print (\"x shape:\", x.shape)\n",
        "print (\"x size: \", x.size)\n",
        "print (\"x dtype: \", x.dtype)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[1 2]\n",
            " [3 4]]\n",
            "x ndim:  2\n",
            "x shape: (2, 2)\n",
            "x size:  4\n",
            "x dtype:  int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3O1arODThrj_",
        "colab_type": "code",
        "outputId": "55bccb9d-2990-4dc1-9632-5bf5ca8362c4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "# 3-D Tensor\n",
        "x = np.array([[[1,2],[3,4]],[[5,6],[7,8]]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x ndim: \", x.ndim)\n",
        "print (\"x shape:\", x.shape)\n",
        "print (\"x size: \", x.size)\n",
        "print (\"x dtype: \", x.dtype)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[[1 2]\n",
            "  [3 4]]\n",
            "\n",
            " [[5 6]\n",
            "  [7 8]]]\n",
            "x ndim:  3\n",
            "x shape: (2, 2, 2)\n",
            "x size:  8\n",
            "x dtype:  int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EfeFhsp2CVs5",
        "colab_type": "text"
      },
      "source": [
        "NumPy also comes with several functions that allow us to create tensors quickly."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z2Qf8EKZln9j",
        "colab_type": "code",
        "outputId": "b8d4e69f-c1a8-4980-a0eb-b8c10a24ed7f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "source": [
        "# Functions\n",
        "print (\"np.zeros((2,2)):\\n\", np.zeros((2,2)))\n",
        "print (\"np.ones((2,2)):\\n\", np.ones((2,2)))\n",
        "print (\"np.eye((2)):\\n\", np.eye((2))) # identity matrix \n",
        "print (\"np.random.random((2,2)):\\n\", np.random.random((2,2)))"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "np.zeros((2,2)):\n",
            " [[0. 0.]\n",
            " [0. 0.]]\n",
            "np.ones((2,2)):\n",
            " [[1. 1.]\n",
            " [1. 1.]]\n",
            "np.eye((2)):\n",
            " [[1. 0.]\n",
            " [0. 1.]]\n",
            "np.random.random((2,2)):\n",
            " [[0.19151945 0.62210877]\n",
            " [0.43772774 0.78535858]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVD-MCiCdcV9",
        "colab_type": "text"
      },
      "source": [
        "# Indexing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PmThuBRHCsCA",
        "colab_type": "text"
      },
      "source": [
        "Keep in mind that when indexing the row and column, indices start at 0. And like indexing with lists, we can use negative indices as well (where -1 is the last item)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v9wKr0HOClLp",
        "colab_type": "text"
      },
      "source": [
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/indexing.png\" width=\"300\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vyt36kFOcVDX",
        "colab_type": "code",
        "outputId": "965ab808-41d1-4ac9-d483-abefede65b9d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Indexing\n",
        "x = np.array([1, 2, 3])\n",
        "print (\"x: \", x)\n",
        "print (\"x[0]: \", x[0])\n",
        "x[0] = 0\n",
        "print (\"x: \", x)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:  [1 2 3]\n",
            "x[0]:  1\n",
            "x:  [0 2 3]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qxHww0didni6",
        "colab_type": "code",
        "outputId": "8da7780d-4a97-48b2-d318-2057bb2c9050",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Slicing\n",
        "x = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])\n",
        "print (x)\n",
        "print (\"x column 1: \", x[:, 1]) \n",
        "print (\"x row 0: \", x[0, :]) \n",
        "print (\"x rows 0,1 & cols 1,2: \\n\", x[0:2, 1:3]) "
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 1  2  3  4]\n",
            " [ 5  6  7  8]\n",
            " [ 9 10 11 12]]\n",
            "x column 1:  [ 2  6 10]\n",
            "x row 0:  [1 2 3 4]\n",
            "x rows 0,1 & cols 1,2: \n",
            " [[2 3]\n",
            " [6 7]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A52pzB9idyDE",
        "colab_type": "code",
        "outputId": "a191a016-6cf4-442f-9700-df294979c413",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "# Integer array indexing\n",
        "print (x)\n",
        "rows_to_get = np.array([0, 1, 2])\n",
        "print (\"rows_to_get: \", rows_to_get)\n",
        "cols_to_get = np.array([0, 2, 1])\n",
        "print (\"cols_to_get: \", cols_to_get)\n",
        "# Combine sequences above to get values to get\n",
        "print (\"indexed values: \", x[rows_to_get, cols_to_get]) # (0, 0), (1, 2), (2, 1)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 1  2  3  4]\n",
            " [ 5  6  7  8]\n",
            " [ 9 10 11 12]]\n",
            "rows_to_get:  [0 1 2]\n",
            "cols_to_get:  [0 2 1]\n",
            "indexed values:  [ 1  7 10]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_R7O5WsVfDij",
        "colab_type": "code",
        "outputId": "5b5f2f05-4605-4e93-90c2-9b255c63ce1e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "# Boolean array indexing\n",
        "x = np.array([[1, 2], [3, 4], [5, 6]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x > 2:\\n\", x > 2)\n",
        "print (\"x[x > 2]:\\n\", x[x > 2])"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[1 2]\n",
            " [3 4]\n",
            " [5 6]]\n",
            "x > 2:\n",
            " [[False False]\n",
            " [ True  True]\n",
            " [ True  True]]\n",
            "x[x > 2]:\n",
            " [3 4 5 6]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "77RCjrQ8gvYW",
        "colab_type": "text"
      },
      "source": [
        "# Arithmetic\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1UJVcNCLfFrV",
        "colab_type": "code",
        "outputId": "ccdd71d1-5ffd-425b-930b-b133f9b6094b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "# Basic math\n",
        "x = np.array([[1,2], [3,4]], dtype=np.float64)\n",
        "y = np.array([[1,2], [3,4]], dtype=np.float64)\n",
        "print (\"x + y:\\n\", np.add(x, y)) # or x + y\n",
        "print (\"x - y:\\n\", np.subtract(x, y)) # or x - y\n",
        "print (\"x * y:\\n\", np.multiply(x, y)) # or x * y"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x + y:\n",
            " [[2. 4.]\n",
            " [6. 8.]]\n",
            "x - y:\n",
            " [[0. 0.]\n",
            " [0. 0.]]\n",
            "x * y:\n",
            " [[ 1.  4.]\n",
            " [ 9. 16.]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RhEuhgrxf11s",
        "colab_type": "text"
      },
      "source": [
        "### Dot product"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P_8JZmIuGDjN",
        "colab_type": "text"
      },
      "source": [
        "One of the most common NumPy operations we’ll use in machine learning is matrix multiplication using the dot product. We take the rows of our first matrix (2) and the columns of our second matrix (2) to determine the dot product, giving us an output of `[2 X 2]`. The only requirement is that the inside dimensions match, in this case the first matrix has 3 columns and the second matrix has 3 rows. \n",
        "\n",
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/dot.gif\" width=\"450\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XyZVF6gXhTWd",
        "colab_type": "code",
        "outputId": "fe001321-aae8-4b84-fb29-05e22f8955d2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Dot product\n",
        "a = np.array([[1,2,3], [4,5,6]], dtype=np.float64) # we can specify dtype\n",
        "b = np.array([[7,8], [9,10], [11, 12]], dtype=np.float64)\n",
        "c = a.dot(b)\n",
        "print (f\"{a.shape} · {b.shape} = {c.shape}\")\n",
        "print (c)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(2, 3) · (3, 2) = (2, 2)\n",
            "[[ 58.  64.]\n",
            " [139. 154.]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EjzDdGLZf3fV",
        "colab_type": "text"
      },
      "source": [
        "### Axis operations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J0Pflg2RIpeR",
        "colab_type": "text"
      },
      "source": [
        "We can also do operations across a specific axis.\n",
        "\n",
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/axis.gif\" width=\"450\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7pB-H-7phsku",
        "colab_type": "code",
        "outputId": "e5c425fa-60dc-4f28-c8e1-d68167ce2db6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Sum across a dimension\n",
        "x = np.array([[1,2],[3,4]])\n",
        "print (x)\n",
        "print (\"sum all: \", np.sum(x)) # adds all elements\n",
        "print (\"sum axis=0: \", np.sum(x, axis=0)) # sum across rows\n",
        "print (\"sum axis=1: \", np.sum(x, axis=1)) # sum across columns"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[1 2]\n",
            " [3 4]]\n",
            "sum all:  10\n",
            "sum axis=0:  [4 6]\n",
            "sum axis=1:  [3 7]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VC379NrkkNHY",
        "colab_type": "code",
        "outputId": "b11ff5c7-6490-4015-c108-400a4b7eb826",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Min/max\n",
        "x = np.array([[1,2,3], [4,5,6]])\n",
        "print (\"min: \", x.min())\n",
        "print (\"max: \", x.max())\n",
        "print (\"min axis=0: \", x.min(axis=0))\n",
        "print (\"min axis=1: \", x.min(axis=1))"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "min:  1\n",
            "max:  6\n",
            "min axis=0:  [1 2 3]\n",
            "min axis=1:  [1 4]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WPlg42K3f6Q8",
        "colab_type": "text"
      },
      "source": [
        "### Broadcasting"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "luY3wfQ2JRPM",
        "colab_type": "text"
      },
      "source": [
        "Here, we’re adding a vector with a scalar. Their dimensions aren’t compatible as is but how does NumPy still gives us the right result? This is where broadcasting comes in. The scalar is *broadcast* across the vector so that they have compatible shapes.\n",
        "\n",
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/broadcasting.png\" width=\"300\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "awQaYigfOFru",
        "colab_type": "code",
        "outputId": "79d4408a-9f2d-4187-a0cd-3e8826b9e1b3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "# Broadcasting\n",
        "x = np.array([1,2]) # vector\n",
        "y = np.array(3) # scalar\n",
        "z = x + y\n",
        "print (\"z:\\n\", z)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "z:\n",
            " [4 5]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KdPKVKtwkWnw",
        "colab_type": "text"
      },
      "source": [
        "# Advanced"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1XM9PXvpf8ep",
        "colab_type": "text"
      },
      "source": [
        "### Transposing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bh0wGW1VNkTz",
        "colab_type": "text"
      },
      "source": [
        "We often need to change the dimensions of our tensors for operations like the dot product. If we need to switch two dimensions, we can transpose \n",
        "the tensor.\n",
        "\n",
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/transpose.png\" width=\"400\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-WEL6iWUNgAo",
        "colab_type": "code",
        "outputId": "09c7837b-0801-4e5a-eed6-b609048d20ba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "# Transposing\n",
        "x = np.array([[1,2,3], [4,5,6]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x.shape: \", x.shape)\n",
        "y = np.transpose(x, (1,0)) # flip dimensions at index 0 and 1\n",
        "print (\"y:\\n\", y)\n",
        "print (\"y.shape: \", y.shape)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[1 2 3]\n",
            " [4 5 6]]\n",
            "x.shape:  (2, 3)\n",
            "y:\n",
            " [[1 4]\n",
            " [2 5]\n",
            " [3 6]]\n",
            "y.shape:  (3, 2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "85HFCOyWgBob",
        "colab_type": "text"
      },
      "source": [
        "### Reshaping"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M0IPD9lEN9mZ",
        "colab_type": "text"
      },
      "source": [
        "Sometimes, we'll need to alter the dimensions of the matrix. Reshaping allows us to transform a tensor into different permissible shapes -- our reshaped tensor has the same amount of values in the tensor. (1X6 = 2X3). We can also use `-1` on a dimension and NumPy will infer the dimension based on our input tensor.\n",
        "\n",
        "The way reshape works is by looking at each dimension of the new tensor and separating our original tensor into that many units. So here the dimension at index 0 of the new tensor is 2 so we divide our original tensor into 2 units, and each of those has 3 values.\n",
        "\n",
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/reshape.png\" width=\"450\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RdEHrnMTnO6k",
        "colab_type": "code",
        "outputId": "a2960f6d-b1f2-464e-bc4c-eba421a29e35",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "source": [
        "# Reshaping\n",
        "x = np.array([[1,2,3,4,5,6]])\n",
        "print (x)\n",
        "print (\"x.shape: \", x.shape)\n",
        "y = np.reshape(x, (2, 3))\n",
        "print (\"y: \\n\", y)\n",
        "print (\"y.shape: \", y.shape)\n",
        "z = np.reshape(x, (2, -1))\n",
        "print (\"z: \\n\", z)\n",
        "print (\"z.shape: \", z.shape)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[1 2 3 4 5 6]]\n",
            "x.shape:  (1, 6)\n",
            "y: \n",
            " [[1 2 3]\n",
            " [4 5 6]]\n",
            "y.shape:  (2, 3)\n",
            "z: \n",
            " [[1 2 3]\n",
            " [4 5 6]]\n",
            "z.shape:  (2, 3)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5qFDGC0oiSM8",
        "colab_type": "text"
      },
      "source": [
        "### Unintended reshaping"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iSvltStjSQk6",
        "colab_type": "text"
      },
      "source": [
        "Though reshaping is very convenient to manipulate tensors, we must be careful of their pitfalls as well. Let's look at the example below. Suppose we have `x`, which has the shape `[2 X 3 X 4]`. \n",
        "```\n",
        "[[[ 1  1  1  1]\n",
        "  [ 2  2  2  2]\n",
        "  [ 3  3  3  3]]\n",
        " [[10 10 10 10]\n",
        "  [20 20 20 20]\n",
        "  [30 30 30 30]]]\n",
        "```\n",
        "We want to reshape x so that it has shape `[3 X 8]` which we'll get by moving the dimension at index 0 to become the dimension at index 1 and then combining the last two dimensions. But when we do this, we want our output \n",
        "\n",
        "to look like:\n",
        "✅\n",
        "```\n",
        "[[ 1  1  1  1 10 10 10 10]\n",
        " [ 2  2  2  2 20 20 20 20]\n",
        " [ 3  3  3  3 30 30 30 30]]\n",
        "```\n",
        "and not like:\n",
        "❌\n",
        "```\n",
        "[[ 1  1  1  1  2  2  2  2]\n",
        " [ 3  3  3  3 10 10 10 10]\n",
        " [20 20 20 20 30 30 30 30]]\n",
        " ```\n",
        "even though they both have the same shape `[3X8]`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TjX41EXUSZVF",
        "colab_type": "code",
        "outputId": "a9bbd6db-3e8e-43c3-faf6-e1080227034d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "x = np.array([[[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3]],\n",
        "              [[10, 10, 10, 10], [20, 20, 20, 20], [30, 30, 30, 30]]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x.shape: \", x.shape)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[[ 1  1  1  1]\n",
            "  [ 2  2  2  2]\n",
            "  [ 3  3  3  3]]\n",
            "\n",
            " [[10 10 10 10]\n",
            "  [20 20 20 20]\n",
            "  [30 30 30 30]]]\n",
            "x.shape:  (2, 3, 4)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8l7Yb2FeEBEQ",
        "colab_type": "text"
      },
      "source": [
        "When we naively do a reshape, we get the right shape but the values are not what we're looking for."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kPA5F3ZkM-Bg",
        "colab_type": "text"
      },
      "source": [
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/reshape_wrong.png\" width=\"600\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0geomcgyZtcz",
        "colab_type": "code",
        "outputId": "2344c67f-f0e8-4a2d-cf32-9396b06c8956",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Unintended reshaping\n",
        "z_incorrect = np.reshape(x, (x.shape[1], -1))\n",
        "print (\"z_incorrect:\\n\", z_incorrect)\n",
        "print (\"z_incorrect.shape: \", z_incorrect.shape)"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "z_incorrect:\n",
            " [[ 1  1  1  1  2  2  2  2]\n",
            " [ 3  3  3  3 10 10 10 10]\n",
            " [20 20 20 20 30 30 30 30]]\n",
            "z_incorrect.shape:  (3, 8)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sIYPoTPhD8of",
        "colab_type": "text"
      },
      "source": [
        "Instead, if we transpose the tensor and then do a reshape, we get our desired tensor. Transpose allows us to put our two vectors that we want to combine together and then we use reshape to join them together.\n",
        "Always create a dummy example like this when you’re unsure about reshaping. Blindly going by the tensor shape can lead to lots of issues downstream."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6cym1KWlNCbB",
        "colab_type": "text"
      },
      "source": [
        "<div align=\"left\">\n",
        "<img src=\"https://raw.githubusercontent.com/practicalAI/images/master/basic_ml/02_Numpy/reshape_right.png\" width=\"600\">\n",
        "</div>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nu-DgEDoacwc",
        "colab_type": "code",
        "outputId": "94dd18d9-45a0-463c-c188-a250c8fe9ff9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272
        }
      },
      "source": [
        "# Intended reshaping\n",
        "y = np.transpose(x, (1,0,2))\n",
        "print (\"y:\\n\", y)\n",
        "print (\"y.shape: \", y.shape)\n",
        "z_correct = np.reshape(y, (y.shape[0], -1))\n",
        "print (\"z_correct:\\n\", z_correct)\n",
        "print (\"z_correct.shape: \", z_correct.shape)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "y:\n",
            " [[[ 1  1  1  1]\n",
            "  [10 10 10 10]]\n",
            "\n",
            " [[ 2  2  2  2]\n",
            "  [20 20 20 20]]\n",
            "\n",
            " [[ 3  3  3  3]\n",
            "  [30 30 30 30]]]\n",
            "y.shape:  (3, 2, 4)\n",
            "z_correct:\n",
            " [[ 1  1  1  1 10 10 10 10]\n",
            " [ 2  2  2  2 20 20 20 20]\n",
            " [ 3  3  3  3 30 30 30 30]]\n",
            "z_correct.shape:  (3, 8)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bf8F5J4ggVR1",
        "colab_type": "text"
      },
      "source": [
        "### Adding/removing dimensions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TlSaJRFaSazJ",
        "colab_type": "text"
      },
      "source": [
        "We can also easily add and remove dimensions to our tensors and we'll want to do this to make tensors compatible for certain operations."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LNYJRMF4qvXN",
        "colab_type": "code",
        "outputId": "d556e030-d619-47df-ad30-c31bce13918a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "# Adding dimensions\n",
        "x = np.array([[1,2,3],[4,5,6]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x.shape: \", x.shape)\n",
        "y = np.expand_dims(x, 1) # expand dim 1\n",
        "print (\"y: \\n\", y)\n",
        "print (\"y.shape: \", y.shape)   # notice extra set of brackets are added"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[1 2 3]\n",
            " [4 5 6]]\n",
            "x.shape:  (2, 3)\n",
            "y: \n",
            " [[[1 2 3]]\n",
            "\n",
            " [[4 5 6]]]\n",
            "y.shape:  (2, 1, 3)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tE1BmoJuns70",
        "colab_type": "code",
        "outputId": "8478e9d7-0a3f-45b2-df84-43755359be19",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "# Removing dimensions\n",
        "x = np.array([[[1,2,3]],[[4,5,6]]])\n",
        "print (\"x:\\n\", x)\n",
        "print (\"x.shape: \", x.shape)\n",
        "y = np.squeeze(x, 1) # squeeze dim 1\n",
        "print (\"y: \\n\", y)\n",
        "print (\"y.shape: \", y.shape)  # notice extra set of brackets are gone"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x:\n",
            " [[[1 2 3]]\n",
            "\n",
            " [[4 5 6]]]\n",
            "x.shape:  (2, 1, 3)\n",
            "y: \n",
            " [[1 2 3]\n",
            " [4 5 6]]\n",
            "y.shape:  (2, 3)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XthM4y7SotAH",
        "colab_type": "text"
      },
      "source": [
        "# Additional resources"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3KmESFstrbFS",
        "colab_type": "text"
      },
      "source": [
        "* **NumPy reference manual**: We don't have to memorize anything here and we will be taking a closer look at NumPy in the later lessons. If you want to learn more checkout the [NumPy reference manual](https://docs.scipy.org/doc/numpy-1.15.1/reference/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4qV1ZXx8CFy8",
        "colab_type": "text"
      },
      "source": [
        "---\n",
        "<div align=\"center\">\n",
        "\n",
        "Subscribe to our <a href=\"https://practicalai.me/#newsletter\">newsletter</a> and follow us on social media to get the latest updates!\n",
        "\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://github.com/practicalAI/practicalAI\">\n",
        "              <img src=\"https://img.shields.io/github/stars/practicalAI/practicalAI.svg?style=social&label=Star\"></a>&nbsp;\n",
        "            <a class=\"ai-header-badge\" target=\"_blank\" href=\"https://www.linkedin.com/company/madewithml\">\n",
        "              <img src=\"https://img.shields.io/badge/style--5eba00.svg?label=LinkedIn&logo=linkedin&style=social\"></a>&nbsp;\n",
        "            <a class=\"ai-header-badge\" target=\"_blank\" href=\"https://twitter.com/madewithml\">\n",
        "              <img src=\"https://img.shields.io/twitter/follow/madewithml.svg?label=Follow&style=social\">\n",
        "            </a>\n",
        "              </div>\n",
        "\n",
        "</div>"
      ]
    }
  ]
}